Papers with HuggingFace Transformers library
Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering (2023.tacl-1)
Copied to clipboard
Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, Suranga Nanayakkara
| Challenge: | Retrieval Augment Generation (RAG) has been developed for use in open-domain question answering (ODQA) but it is not optimized for use with other specialized domains such as healthcare and news. |
| Approach: | They propose an extension to RAG that can adapt to a domain-specific knowledge base by updating all components of the external knowledge base during training. |
| Outcome: | The proposed extension can adapt to a domain-specific knowledge base by updating all components of the external knowledge base during training. |
AdapterHub: A Framework for Adapting Transformers (2020.emnlp-demos)
Copied to clipboard
Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, Iryna Gurevych
| Challenge: | AdapterHub framework enables dynamic “stiching-in” of pre-trained adapters for different tasks and languages. |
| Approach: | They propose a framework that allows dynamic "stiching-in" of pre-trained adapters for different tasks and languages. |
| Outcome: | The proposed framework allows dynamic “stiching-in” of pre-trained adapters for different tasks and languages. |
YANMTT: Yet Another Neural Machine Translation Toolkit (2023.acl-demo)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is an end-to-end approach that provides stateof-the-art results for a variety of language pairs. |
| Approach: | They propose to build an open-source neural machine translation toolkit on top of HuggingFace's Transformers library and use it for pre-training and fine-tuning sequence-to-sequence models. |
| Outcome: | The proposed toolkit is built on top of the HuggingFace Transformers library and provides advanced features such as document/multi-source NMT, simultaneous NMT and mixtures-of-experts. |